NASA-CR-200216 


FINAL TECHNICAL REPORT 


/ /u-o^/ 

cP 



"Enroute Flight Planning: Evaluating Design Concepts for the Development 
of Cooperative Problem-Solving Systems * 1 


NAS A- Ames Cooperative Agreement No. NCC 2-615 


Philip J. Smith 

Cognitive Systems Engineering Laboratory 
Department of Industrial and Systems Engineering 
The Ohio State University ^ 

210 Baker Systems, 1971 Neil Avenue 
Columbus, OH 43210 


(NASA-CR-200216) ENROUTE FLIGHT 
PLANNING: EVALUATING DESIGN 
CONCEPTS FOR THE DEVELOPMENT OF 
COOPERATIVE PROBLEM-SOLVING SYSTEMS 
Final Report (Ohio State Univ.) 

34 p 


<0 


8 /> 


N96-19043 


Unci as 


G3/04 0100678 



Design Concepts for the Development of Cooperative 
Problem-Solving Systems 


Philip J. Smith 

Cognitive Systems Engineering Laboratory 
Department of Industrial and Systems Engineering 
The Ohio State University 
210 Baker Systems, 1971 Neil Avenue 
Columbus, OH 43210 


Elaine McCoy 
Department of Aviation 
Aviation Institute 
University of Nebraska at Omaha 


Chuck Layton 

Galaxy Scientific Corporation 
Atlanta, G A 


TomBihari 

Adaptive Machine Technologies, Inc. 
Columbus, OH 



Abstract 


There are many problem-solving tasks that are too complex to fully automate given the current state 
of technology. Nevertheless, significant improvements in overall system performance could 
result from the introduction of well-designed computer aids. 

We have been studying the development of cognitive tools for one such problem-solving task, 
enroute flight path planning for commercial airlines. Our goal has been two-fold. First, we have 
been developing specific system designs to help with this important practical problem. Second, we 
have been using this context to explore general design concepts to guide in the development of 
cooperative problem-solving systems. These design concepts are described below, along with 
illustrations of their application. 



The Application Area 


Before take-off, a complete flight plan is developed describing the route, altitudes and speeds that a 
commercial airliner is expected to follow in flying from its origin to its destination (Rossmore, 
1991). This initial flight plan is rarely followed exactly as specified prior to take-off, however. 
Minor amendments to the plan are common; major changes are not at all unusual. 

Such replanning of the flight while enroute arises because of the dynamic, unpredictable nature of 
the “world” that must be dealt with. Weather patterns do not always develop as predicted, 
resulting in unexpected areas of turbulence, less favorable winds or storms that must be avoided. 
Air traffic congestion may delay take-off or restrict the plane to lower than planned altitudes while 
enroute. Airport or runway closures can cause major disruptions. Mechanical failures, medical 
emergencies or other critical problems may force the plane to divert to a nearby airport 

A Problem Space Bessription 

Enroute flight planning can be represented as search through a problem space (Laird, Newell and 
Rosenbloom, 1987). When some problem arises, as described above, the flight crew, Dispatch 
and Air Traffic Control must develop a revision of the flight plan. To generate this revised plan, a 
variety of alternative solution paths may be considered. 

A state description for one of the possible problem space descriptions consists of : 

1 . The plane’s current location (a point along its route and an altitude) and airspeed; 

2 . The plane’ s currently approved flight plan; 

3 . Static and dynamic characteristics of the plane such as its weight (which changes as fuel 
is consumed), its maximum altitude capabilities (which change as a function of the 
plane’s weight and airspeed), its fuel consumption characteristics, etc. Characteristics 
that are normally considered static may in some cases change because of a problem like 
engine failure; 

4. Actual and forecast weather along the plane’s current path and any possible alternative 
paths. The state description needs to include measures of uncertainty about weather 
forecasts, as well as the best “guess”; 

5 . Information on passenger connections and flight crew availabilities; 

6 . Static and dynamic characteristics of airports that could be used for landing (runway 
lengths, visibility, air traffic congestion, etc.); 



7 . Similar information for any other planes whose paths could interact with possible 
alternative paths for the plane that is the focus of the replanning activities. 

(This is a simplified summary of a state description. Each of these components is actually 
composed of many additional elements.) 

Major operators include: 

1. Changing altitude; 

2. Changing airspeed; 

3. Changing the route; 

4 . Changing the destination (a special but important ease of changing the route). 

Each of these operators can be applied to either the plane that is the primary focus, or some other 
plane that its plan interacts with. Furthermore, the first three operators can be applied to different 
segments of the flight. The plane may fly at 33,000 feet from Milwaukee to Chicago, but at 
25,000 feet from Chicago to Toledo. 

There are also a number of constraints. Planes must maintain a certain separation distance (to 
comply with FAA regulations). Planes often are required to fly along “highways in the sky”. 
(They often fly along a sequence of navigational fixes to get to some destination, instead of flying 
straight to that point. They are also constrained to fly at certain altitudes. Over the continental 
U.S., for instance, 33,000 feet is an “eastbound only” altitude.) There are also certain physical 
limitations. The plane can’t fly if it is out of fuel and it can’t land at an airport where the runways 
are too short. Some of these constraints are actually “soft”. If, for instance, there is no traffic, Air 
Traffic Control (ATC) may allow the plane to fly west at an “eastbound only” altitude. Similarly, 
ATC may appirive a vector that deviates from the jetroutes in order to avoid a storm or save on 
fuel. 

Description of the state spaces, operators and constraints are difficult because there are so many 
possibilities to consider. Definition of the evaluation function for selecting among operators is 
even more challenging, however. It is clear that multiple competing and complementary goals are 
considered (Wilensky, 1983) in evaluating preferences among the alternative operators (or operator 
sequences). Safety, fuel consumption, time and passenger comfort are all important 
considerations. It is not so clear, though, exactly how human planners currently deal with 
tradeoffs among these goals. 



In short, the full problem space for enroute flight planning is very large and complex. Multiple 
goals must be considered in a highly stochastic environment where multiple plans must be 
coordinated. 


Cooperative ProbtemrSplving as a Conceptual, Approach 

Our conclusion, based on this initial problem space analysis, was that complete automation is not 
likely to be an acceptable approach for improving such planning performance (Boehm-Davis et al., 
1983; Charles and Nagel, 1985; Price, 1985). Neither knowledge-based systems techniques nor 
optimization methods are sufficiently trustworthy to replace human judgement on such a planning 
task. Concerns over inappropriate model formulation, incompleteness of the knowledge-base, 
brittleness in dealing with novel situations, difficulties in trading off among competing goals and 
inadequacies when making decisions in uncertain environments all introduce significant objections 
to full automation as a solution. 

One approach to alleviating such concerns is to try to build better computerized problem-solvers. 
Current work on “deep reasoning’ systems and qualitative modeling falls into this tradition (Davis, 
1984; Sembugamoorthy and Chandrasekeran, 1986). 

An alternative (but complementary) approach is to focus on shared problem-solving (Robertson, 
Zachery and Black, 1990). This, in fact, was one of the major motivations behind much of the 
early work on expert systems. In response to failures and lack of acceptance for problem-solving 
systems based on optimization techniques, the artificial intelligence community suggested the 
design of systems that solve problems “like people do”. Michie (1986), for example, states: 

“The suggestion is that reliance on the escalating power of brute force may be heading 
towards danger. However effective and reliable such systems may be in normal 
conditions, use of brute force may not be worth the price paid during the rare episodes 
when a computer-controlled power station or military installation or air-traffic control 
system malfunctions. On these occasions, a new factor becomes paramount. The human 
operator or supervisor needs to follow what the computing system thinks its doing.” 

Early work on expert systems, as a reaction to optimization approaches, set out to increase the 
cognitive compatibility of computer problem-solvers and their users by attempting to mimic human 
cognitive processes. This is only one of many concepts, however, that is useful in guiding in the 



design of more effective cooperative problem-solving systems. 


Below, we describe additional design concepts that have guided our work in exploring the design 
of a cooperative planning system. Equally important, we illustrate the importance of understanding 
not only how people correctly solve particular kinds of problems (Hayes-Roth and Hayes-Roth, 
1979; Smith, Fraser, et al., 1991), but also the nature and causes of errors that people make in 
solving these problems (Fraser, Smith and Smith, 1992; Nagel, 1988), and the ways in which 
alternative system designs influence and enhance shared problem-solving. At this point these 
design concepts should be viewed as hypotheses to be tested. Our flight planning testbed offers a 
context for conducting such empirical tests. 

Initial Studies 

In order to better understand human performance on flight planning tasks, we began by: 

1 . Interviewing pilots, air traffic controllers and dispatchers. (Dispatchers work for 
individual airlines and are responsible for developing the original plans for a flight and 
for helping the flight crew generate amendments while enroute); 

2 . Conducting a survey of 1 36 pilots to identify situations where they had experienced 
problems with enroute flight planning; 

3 . Running studies in a flight simulator to observe actual flight planning activities (Galdes 
and Smith, 1990). 

These studies made it clear that enroute flight planning activities are currently distributed among the 
flight crew, ATC and Dispatch. They also made it apparent that, at present, all three parties play a 
major role in detecting situations that require replanning, in generating possible flight amendments 
and in evaluating the alternative plans. Examples of behaviors observed in our simulation study 
are given below. 

Example I 

Fifteen minutes after takeoff, the pilot requested clearance to climb from FL 250 to FL 290. ATC 
denied this request because of other traffic, hi response to this event, the flight crew did the 
following; 

1 , Asked ATC how long they would be held at FL 250. 

2 . Noted that they “ought to call Dispatch and tell them we’re at a different altitude”, but 



chose not to call Dispatch yet 

3 . Asked themselves: “What do you think our difference in bum would be at 250?” 

4. Determined the differences in fuel bum and time (actual vs. planned) at the next 
navigational fix: “47.4-we’re 200 pounds under.” 

5 . Checked the wind speeds and direction: “Have the winds changed at all? We’re 
coming up on Mustang, Mustang has winds at 290 of 44 knots, 

6 . Predicted the extra fuel bum resulting from staying at FL 250 until Battle Mountain (the 
point at which ATC had indicated they could probably climb): “I guess we know we’re 
going to bum some more fuel staying down here, but probably as much as 500 pounds 
maybe.” 

7 . Further evaluated the implications of staying at FL 250: “Twenty-five minutes down 
here. That’ll let us get to 33 a little ahead of time because we’ll have burned off fuel 
just a little ahead of time. Yeah. Possible. I don’t know.” 

8. Planned their next change in path: “Battle Mountain. That’s when I’m hoping to get 
29,000.” 

9 . Evaluated this plan by checking the winds at Battle Mountain. 

As this example illustrates, the flight crew was extremely active in considering alternative flight 
paths. They collected a variety of data to determine the implications of the unplanned deviation 
from their route, and to decide what they should do next. Some of this data involved comparing 
actual performance (e.g., fuel bum) with that expected under the original plan. Other data required 
making predictions about future performance if the current altitude was maintained. 

Ex am ple.! 

In the first example, ATC instructions made it necessary for the pilots to consider the implication of 
a different route. In this second example which occurred 54 minutes into the flight simulation, one 
crew detected data that caused'them to consider a different route for other reasons: 

1 . Looking at a radar display, the co-pilot noted: “We could have some activity on the 
way to Detroit, too. I think we’re going to want to go north of that North or south. It 
looks like north would be better.” 

2. The crew then proceeded to develop such a plan: “It seems like maybe we could 
reroute our flight up above there [North] rather then wait ‘til we get up here... What 
kinds of VORs are we looking at then? Should we maybe go to Aberdeen flying up 
north and possibly Redwood Falls?” 



3 . The pilot then requested such a change: “We have a routing request we’d like to have 
you pass on to our dispatcher. We’d like to fly Jet 32 to Aberdeen, then Jet 70 to 
Badger. We’d like to remain at FL 250 for the time being.” 

This example again illustrates the fact that the flight crew can play an active role in detecting the 
need to consider an alternative plan and in generating the alternative plan. 

Example 3 

Two hours and sixteen minutes into the flight, another crew reacted to a storm they were 
encountering: 

“That looks kinda nasty. We tried to tell them a long time ago we wanted to go north of 
that. I’m not wild about going between those things. There’s not 20 miles between them. 

I vote total deviation. Ask’em for a vector around the north side of the weather. How far 
are we going to have to go? 100 miles? If we start down, we don’t have to go as far out of 
our way. Just tell ’em we want to vector north of the weather and let them [ATC] do it. 

We don’t have enough information to be that specific. There’s no way we’re going to fly 
into that...Holy shit! There’s stuff behind it, too. Holy Mother!” 

This example provides a nice illustration of the role of the crew in detecting a problem and 
considering alternatives. It also points out the importance of coordination between the crew, ATC 
and Dispatch. In particular, the crew noted: “Taking our deviation a lot further back would have 
made a whole lot more sense.” 

Example 4 

Two hours and forty-eight minutes into the flight, one crew began to worry about their destination: 

“I have a bad feeling about Detroit Should have been starting to clear... The minimum 
there - we need a half mile... What did they show for the fuel there? 18.6 - One thousand 
pounds less than original... I recommend, gentlemen, if Detroit doesn’t look good we go 
direct to Cleveland and we go to the 100 Bomb Group for dinner, to the restaurant right 
next to the airport... Chicago’s pretty good. Milwaukee’s not bad. Our landing fuel just 
gets lower and lower.” 



Based on such data, and on the results of our interviews and surveys, we completed a cognitive 
task analysis (Galdes and Smith, 1990). This identified pertinent goals, data and problem-solving 
activities, as well as providing insight into the roles of the various players. It also identified 
problems arising in existing planning environments, ranging from failures to detect problems with 
the current flight plan in a timely manner, to inadequate generation of alternative solutions (thus 
missing a good alternative), to fixation on a potentially dangerous solution. 

We then used this analysis as the basis for exploring alternative system designs in the Flight 
Planning Testbed (FPT), a prototyping environment to study the design of tools to aid in enroute 
flight planning. Below we: 

1 . Describe the prototyping environment built to support system development and 
testing: 

2 . Present specific implementations explored using FPT; 

3 . Discuss general design concepts that guided us in the development of this cooperative 
problem-solving system. 

As part of these discussions, we also point out important insights that arose from our cognitive 
task analysis. 

It should be noted that all the design concepts studied in FPT are potentially applicable to the 
design of tools to aid dispatchers in their pre-flight and enroute planning activities. As datalink 
capabilities are enhanced, some of them may also be useful in developing on-board support 
systems. 


Development of a Prototyping Tool and Design Concepts 

To study the design of planning tools, we have developed a prototyping tool that can support the 
development and testing of a variety of design concepts. This prototyping shell, designed to run 
on a Mac n, provides a general environment for developing application software, but does not 
inhibit programmers from modifying the environment if necessary. Written in C, the system can 
control displays on up to six color monitors. 


This prototyping tool supports the creation and use of multiple window displays on each screen 
and the use of both mouse and keyboard inputs. The tool also provides both real-time and 
simulation-time clocks to control the timing of events and to record response times. The system 



records the time and nature of all actions made by a subject, and can replay the entirety of a 
subject’s actions at a later time. 


Using this prototyping tool, we have been exploring a number of design concepts. Important 
features are described below. 

Map Display 

The system is capable of generating an accurate map display for any portion of the world. To 
accomplish this, we have ported to the Mac II a program (and associated database) that can produce 
accurate displays of any portion of the world, using any one of several available map projections. 

The system also allows for easy, rapid display of weather information on this map display. By 
simply pressing buttons with a mouse, the planner can select a variety of weather overlays (radar 
weather, jet streams, fronts, etc.) to display on the map. (See Figure 1). In this manner, the 
planners (dispatchers or pilots) can personalize the weather display to meet their current needs. 
Furthermore, by double-clicking with the mouse on any portion of the map display, the planner 
can zoom in on the region, seeing a close-up display. 


Insert Figure 1 about here 


In order to facilitate viewing trend information, the planner can also view weather sequences over 
time on the map display. This is accomplished be moving the plane along its route on the map. 

The plane is moved using a scroll bar controlled by the mouse. The map display can also show 
weather infonrfetion at different altitudes. (It is predicted that such data will be available in the next 
few years.) 

In addition to presenting weather information, the map display can show up to four alternative 
routes for the plane. It also displays the location of the plane on the active route. Both the plane’s 
location and the weather displays are updated over time during the simulation. 

Routes can be created or changed on the map display in three ways. One way is by sketching 
routes on the map itself using the mouse. The planner can create new legs off an existing path or 
create a totally new route. A second way to create or change routes is described in the section on 



the Route Information Display. In that window, changes to routes can be made using the 
keyboard. (Informal evaluations have indicated, though, that everyone prefers to create new routes 
graphically using the mouse.) 

A third way to create routes is to set constraints on the desired solution (things like maximum 
turbulence levels, maximum precipitation levels or desired destination) and to then ask the 
computer to work out the details. (See window in the lower right hand side of Figure 2.) In FPT, 
the computer does so by using linear programming techniques to search for a plan (a sequence of 
navigational fixes, altitude profile and speeds) that, subject to the constraints set by the user, 
minimizes fuel consumption (taking wind components into account). 


Inset Figure 2 about here 


Communications Window 

In addition to the map display, this system design has a window that provides a text editing 
environment for preparing and sending written messages to other parties involved in the planning 
activities. Routes drawn by a dispatcher on the Map Display, for instance, could be transmitted to 
the flight crew along with text (This assumes the existence of suitable datalink capabilities.) 

Airport Information Window 

This window displays both static information (number of runways, etc.) and changing information 
(weather, NOT AMS, etc.) about specific airports. The planner can request such information by 
typing in the airport’s identifier or by scrolling through an alphabetical list and selecting the airport 
with the mouse* 


Route Infor mation Display 

The Map Display provides a graphic presentation of weather data. There are other types of 
information, however, that are better displayed in a text format. We have developed a spreadsheet 
concept to present such information. Figure 3 shows the spreadsheet display available in the 
system. Several important features are illustrated. First, the layout of data in the form of a 
spreadsheet seems well suited to this application. The horizontal sequence of information on the 



spreadsheet corresponds to the horizontal sequence of navigational fixes and jet routes along the 
flight path. Information specific to particular navigational fixes and jet routes is displayed under 
the column with the corresponding navigational fix or jet route label. 


Insert Figure 3 about here 


Second, the spreadsheet allows the planner to immediately view the implications of changes in the 
flight plan. The planner can make changes in the plane’s route on the spreadsheet by simply 
adding or deleting the appropriate navigational fix. These changes in the route are immediately 
drawn on the Map Display. (Alternatively, the planner can change the route by direct manipulation 
of the paths shown on the Map Display. These changes are propagated to the spreadsheet.) The 
planner can also make changes in the planned altitudes and airspeeds on the spreadsheet. 

When a change is made in the flight plan, the system appropriately changes the other information 
displayed (such as arrival time and fuel consumption). The spreadsheet allows the planner to view 
a variety of such information, such as wind components and distances between navigational fixes, 
as well as fuel consumption and arrival time information. Summary information is provided at the 
bottom of the screen for all routes that have been created, thus facilitating comparisons among 
alternative routes. 

The bottom half of the spreadsheet allows the planner to easily compare information about different 
altitudes along a route. The planner can display information such as turbulence, fuel consumption 
and wind components at these different altitudes. To facilitate such comparisons, the planner can 
display the current altitude profile, minimum fuel profile and maximum altitudes. These kinds of 
information aref displayed graphically within the spreadsheet itself. 


Intelligent Aids 

In addition to providing integrated access to different types of information, there are four areas 
where the computer could potentially use knowledge to make suggestions: 

1 . Finding a “good” plan, e.g., a “good” route (sequence of navigational fixes), “good” 
altitudes and “good” airspeeds (Hendler, Tate and Drummond, 1990; Korf, 1987; 



Stefik, 1981; Talcott, Marvin andLehner, 1989); 

2. Inferring the intentions of the human planner (Brooks and Lizza, 1991; Billings, 1991; 
Rouse, 1991) in order to facilitate communication; 

3 . Alerting the planner when some important new data is available or when significant 
problems exist with a plan that he or she has proposed (Nagel, 1988); 

4. Helping the planner to find a good alternative destination if the need arises. 

Such capabilities and associated issues are discussed in the context of the design concepts 
presented below. 


Design Concepts 

In studying the design of aids for enroute flight planning (Johannsen and Rouse, 1983; Sexton, 
Banks, et. al., 1981a), we have encountered a number of relevant design concepts that apply. 
These are discussed below. The value of such a list of concepts (and examples of their 
applications) is their ability to stimulate the thinking of system designers. The designer must still 
consider his or her particular context in order to assess the applicability of a particular design 
concept', and to generate ideas on how to apply it to the specific problem area. By considering 
such a list of concepts, however, the designer of some new system may come up with solutions 
that might otherwise be overlooked. 

Concept 1. Use data abstractions to help planners deal effectively with large 
quantities of data. 

In the near future, the amount of information that could be provided to the people responsible for 
enroute flight planning will be greatly increased (Billings and Cheaney, 1981). Data about 
passenger connections, flight crew schedules and air traffic congestion is already available for use. 
In addition, the technology exists to provide detailed, frequently updated weather information. 
Every plane in the sky is a potential weather sensor transmitting data about turbulence, winds, etc. 
to ground stations. (Airlines are already experimenting with this.) In addition, wind profilers, 
NexRad and automated weather stations will be available to provide further detailed weather data. 

Three questions arise: 

1 . What data should actually be provided to planners? 

2 . How should this data be displayed and utilized? 

3 . Who should have access to what data (ATC or Dispatch or the flight crew?) 

In this section we illustrate one answer to the second question. 



Consider a system where a national turbulence map is available and updated regularly. The 
quantity of data to consider is huge. Clearly, the planner for a particular flight can begin by 
focusing attention on the airspace along the flight’s route. With up to 20 flight segments for longer 
flights, however, the number of relevant pieces of data is still very large. 

We need some way to help the planner focus attention on potential problem areas, and on likely 
solutions. Our current design illustrates one solution, using data abstractions. 

Consider the detailed spreadsheet display. The spreadsheet can display turbulence reports for each 
of several altitudes along the route. (Turbulence data could also be displayed graphically on the 
map display.) It also displays the planned and minimum fuel profiles. (The planned altitudes are 
shown in the same color as the route.) 

It would be impossible to display detailed turbulence data (such as “there is light turbulence along 
Jet Route 793 fifty miles east of CMH”) within such a compact display. Indeed, for just one 
individual flight segment, there could be considerable variation in turbulence levels at different 
points. 

We could simply create a listing of all the turbulence information for all of the points along the 
route for all of the nearby altitudes. Instead, we are using our spreadsheet display to present an 
abstraction of this turbulence information. The label (light, moderate, etc.) in the spreadsheet cell 
indicates the maximum turbulence level along that segment at that altitude (see Figure 3.) 

Imagine a planner who wants to ask: 

Am I litely to encounter significant turbulence in the next segment of my flight? 

This planner can simply scan along the altitude profile as displayed in the spreadsheet and see 
whether any of the flight segments shows significant turbulence. If, for instance, one segment 
indicates moderate turbulence, he/she can click on that cell, opening a window which describes in 
detail the nature and extent of the turbulence along that segment 

Imagine this same planner asking: 


Can I avoid this moderate turbulence by changing altitude? 



He/she can simply scan the spreadsheet cells, looking for an altitude corresponding to that flight 
segment that has less turbulence- indicated. (An analogous form of data abstraction applies to the 
map display, where the planner could zoom in on a region and get more detailed information about 
weather and airports.) 

This concept is particularly important in designing cooperative systems. The goal is to allow the 
computer and the human planner to both be actively involved in detecting the need to replan, and in 
generating and evaluating alternative plans. In order to critique the computer’s suggestions and to 
generate alternatives of his/her own, the human planner needs to access the pertinent data and to 
have it displayed in a usable form. It is not sufficient to simply provide the human planner with an 
explanation justifying the computer’s recommendations. The assumption behind the design of a 
cooperative system is that there will be cases where the human planner will be capable of 
generating a better plan than the computer. Data abstractions offer one method for assisting the 
human planner in detecting the need to consider alternatives and in accessing the data necessary to 
accomplish this. 

Concept 2. Allow direct manipulation of graphic displays to enhance 
exploration. 

Our preliminary tests indicate that dispatchers and pilots are very enthusiastic about the ability to 
graphically create and manipulate routes. The ability to sketch the changes directly on the map 
display makes it much easier to explore alternate routes to avoid bad weather. 

Using our map display, the planner can also move the plane along the route and watch the 
(forecast) weather change. This helps the planner to assess trends in the weather and their potential 
impact on the flight. It also helps the planner to answer questions such as: 

Am I likely to encounter bad weather at my destination? 

If the answer to this question is affirmative, the planner may want to request extra fuel (if this 
potential problem has been noted before takeoff) or identify suitable alternate airports. 

Like Concept 1, this design concept recognizes the importance of supporting the human planner in 
developing and evaluating alternative plans. Such uses of direct manipulation (Booth, 1989; 
Norman and Draper, 1986; Sheridan, 1987) make it easier to accomplish this goal by allowing the 
planner to explore alternatives by manipulating routes and altitudes on the data displays 



themselves. 


Concept 3. Support planning and plan evaluation at many levels of detail. 

Sacerdoti (1974) discusses the use of abstraction hierarchies to improve the efficiency of planning 
systems. Based on an analogy to this idea, we have developed a system where the human planner 
can develop plans at several levels of detail. 


Flight planning is well characterized in terms of such an abstraction hierarchy. Imagine, for 
instance, a pilot flying from San Francisco to Detroit who learns of a line of thunderstorms 
crossing his flight path over the Plain States. The primary decision for him, his dispatcher and 
ATC is whether to deviate north or south of this storm. In order to evaluate this choice, however, 
it is necessary to specify additional details. Navigational fixes, altitudes and airspeeds must also be 
specified. 


In order to support this goal: 


1 . The planner indicates that he/she wants a new route that avoids the storm; 

2 . Be default, the computer fills in the lower level of details, finding navigational fixes that 
achieve this goal, and subject to that constraint, minimize fuel consumption; 

3 . The planner then evaluates the details of this solution by looking at the spreadsheet 
displaying route information such as expected arrival time and fuel consumption. If he 
chooses, he can alter the computer’s recommendations for the lower level details and 
compare his choices with the computer’s. 

Consider another situation where the plane encounters turbulence. He/she wants to decide whether 
to go higher or lower. Using the spreadsheet display, the planner can directly generate and 
evaluate alternative altitudes. 


Thus, we have designed a system where: 


1 . Displays exist corresponding to different levels of detail in the planning hierarchy; 

2 . The planner can view and make changes on any of these displays. The planner can 
change navigational fixes on the map display and altitudes or airspeeds on the 
spreadsheet He/she can therefore make changes at any level of detail desired. He/she 
can also look at the data needed to evaluate decisions at that level of detail; 

3 . The computer, by default, handles lower levels of details. The planner can, however, 
compare the computer’s recommendations with his/her own ideas and make changes as 
desired at any level of detail. 

Thus, using this architecture, the planner can easily explore “what if’ questions at any level of 



detail desired. 


Note also, for this part of the system, important issues begin to arise regarding the nature of the 
computer’s planning processes. The planner may initially choose to rely on the computer’s 
solutions at lower levels of detail while deciding whether to select a route north or south of the 
storm (as described above). At some point, however, the planner must decide whether to accept 
these lower level details as suggested by the computer, or to modify them. This need raises 
interesting questions about how the computer should develop its suggestions. (These issues are 
discussed further under Concept 5.) 

Concept 4, Facilitate communication and cooperation by designing a system that 
can infer the planner’s current goals. 

This is a popular suggestion in the literature today. In developing specific design ideas for this 
application, however, we found that it was more effective to provide the planner with a language to 
tell the computer his/her goal directly, rather than expect the computer to infer it The methods for 
doing so are described under Concept 5. 

Concept 5. Be sure there is a clear, easy to understand conceptual model for 
controlling and understanding the computer’s processing. 

The assumption behind building cooperative systems is that two “heads” are better than one, 
especially when one of them is only a computer. This raises some interesting questions: 

1 . Should we try to design the computer so that it thinks “like” people do? 

2. How do we ensure that the human planner and the computer system have the same 
goals and priorities? 

3 . How do we design the system to induce the human planner to play an active role in 
planning rather than relying on the computer to do all the work? 

Lehner and Zkk (1987) present data suggesting that computers need not think exactly like their 
human partners. Indeed, they found that best performance occurred when the computer did not 
use the same reasoning process. A necessary condition for this result, however, was that the 
human partner be able to understand how the computer arrived at its conclusions. 

Several flight planning systems have been developed that use optimization techniques to find the 
“best” plan for a given situation (Sorensen, Waters and Patmore, 1983). To use such systems, the 
planner must assign weights to different factors such as fuel consumption and tardiness. This is 
certainly different from the way humans reason about flight planning (Galdes and Smith, 1990). It 



is also, however, difficult for humans to understand the underlying reasoning. We are 
consequently investigating the development of “cognitive interfaces” to such optimization systems. 

At one extreme is a system that simply finds the “best” route in terms of a single objective, such as 
fuel consumption or arrival time. The human planner is then forced to play a very active role, 
looking at other factors such as turbulence. 

At the other extreme is a system where the human planner can set up constraints for the flight, such 
as: 


1. Minimum acceptable remaining fuel; 

2. Earliest acceptable arrival time; 

3. Latest acceptable arrival time; 

4 . Maximum acceptable turbulence level; 

5 . Minimum clearance from thunderstorms; 

6 . Restriction to an ATC preferred route; 

7. A particular destination. 

Figure 2 illustrates an interface for setting such constraints. The menu in the lower right hand 
comer lets the planner set constraints on things like the maximum turbulence level or the desired 
destination. 

Such constraint setting is more compatible with normal human planning considerations (Galdes 
and Smith, 1990) than asking the person to weigh the relative importance of different factors. 

There is still, however, a need to support independent planning by the person. What if, for 
instance, the plane has pressurization problems and can’t climb to its normal altitudes? What if the 
passengers have just had lunch? What if the nearest accessible alternate airport is further away than 
originally planned because of bad weather? 

* 

Thus, we are using FPT to study the use optimization algorithms and the design of cognitive 
interfaces to these algorithms. We are also, however, studying ways to support independent 
human planning, and studying ways to ensure that such planning will actually occur in a timely 
fashion. 

Finding Alternative Destinations. Similar issues arise in developing aids to help find a new 
destination. One approach is to have the system generate a “best” alternative. This approach, 
however, assumes that the computer knows what “best” is for the particular situation. In some 
cases this will be determined by the time required to get there (as in an acute medical emergency). 



In other cases, it may be determined by a combination of factors such as the degree of traffic 
congestion and the availability of passenger connections. In still other cases, it may be determined 
by the amount of fuel needed to get there. At a minimum, the human planner must know how such 
a system defines “best”, so that he/she will know when to ignore its recommendations. (Even with 
such knowledge, though, the human planner may become overreliant on the system and fail to note 
a problem with its recommendations.) 

An alternative design is to develop a system that the human planner can query, asking questions 
like: 


What airports can this plane reach within an hour? 

What airports can this plane reach with 15,000 pounds of fuel? 
How long will it take to get to ORD? 


Such a design helps to ensure that the human planner takes an active role in the problem-solving as 
he/she must integrate such information in the selection of an alternative destination. It also, of 
course, increases the human planner’s workload. 


Concept 6. Create a microworld in which the person can actively explore “what- 
if” questions and get useful feedback to help in evaluating alternative 
plans. 

The literature on intelligent tutoring systems discusses the use of computer-supported 
“microworlds” to allow students to explore (Wenger, 1987). The same concept is supported in 
FPT. The planner can ask questions like: What if I go north around the storm or fly over it? FPT 
provides feedback regarding fuel consumption, arrival times and turbulence. 


Concept 7. Support a variety of planning “models” to accommodate different 
situations and people. 

In our simulation studies of flight crews, we observed behaviors consistent with several different 
models of planning as the flight crews dealt with a variety of different situations or causes 
requiring replanning. Such causes included: 


1 . The development of areas of turbulence; 

2 . The unexpected formation of localized storms; 

3 . Changes in winds at different altitudes; 

4 . The appearance of other air traffic that prevents planned altitude changes. 

Example 1. In our simulation study, the flight crews noted that they were behind schedule and 



burning up more fuel than expected under the original plan. They concluded that the problem was 
a headwind that was stronger than expected under their original plan. The crews asked ATC 
whether there were any reports on winds at other altitudes. They learned that the headwinds were 
favorable at lower altitudes. They compared the tradeoff between the benefits of the lower 
headwinds and the cost of flying at die lower altitude, and decided it was preferable to fly at a 
lower altitude. They requested clearance from ATC to do so. 

Example 2. Flight crews encountered light to moderate turbulence. They considered changing 
altitudes to avoid it, or slowing the plane to reduce its effects. They checked for pilot reports on 
the likely duration and magnitude of the turbulence at that altitude, and on turbulence levels at other 
altitudes. The turbulence was reported to be very localized, so they decided to ride it out, slowing 
down to reduce its effects. 

Planning Behavior. Our data indicate that, currently, flight crews generally respond to such 
localized disturbances by generating solutions that are minor modifications of the original plan. In 
most cases, the crew doesn’t replan the entire remainder of the flight, they simply select an 
immediate response to the local problem and act on it (after getting approval from ATC). They 
assume that they will be able to find additional minor modifications for the remainder of the flight if 
the need arises (Suchman, 1987). 

Model 1 - Discussion. Three points merit discussion. First, under these circumstances, plans are 
generated by attempting to make minor modifications to the original flight plan. It is assumed that, 
because the modifications are small, the potential implications for later in the flight do not have to 
be considered in detail. It is assumed that any later modifications made necessary by the current 
change will again be minor, and that acceptable modifications will be possible. 

This decentraliied approach to planning makes strong assumptions about the “world”. It assumes 
that the flight plans of different planes are not tightly coupled. It assumes small changes in one 
plane’s plan do not usually result in significant disruptions of other planes’ plans, or of overall 
system performance. It also assumes that the “world” generally allows a variety of small changes 
to be made. Consequently, it is unnecessary to anticipate the availability of future modifications 
that will be made necessary by the current minor modification. It is assumed that some acceptable 
modification will always be available to meet future needs. 

The third point is that, at present, such localized planning is accomplished in one of two ways. 

The first method can be characterized as a simple forward search with a short planning horizon. 



The pilot looks at the immediately available alternatives (changes in altitude, vectoring around the 
storm or turbulence, slowing down to reduce the effects of turbulence, etc.) and picks the one that 
seems to best solve his/her immediate problem. The second method is somewhat analogous to 
case based reasoning (Klein, 1989), except that the pilots access a broader “institutional” memory. 
They ask ATC whether ATC or any other pilots have already found a solution to the immediate 
problem and then make use of that solution (with minor modifications as needed). 

Our present design of FPT supports such decentralized, localized planning. The planner can use 
the map display to find a set of navigational fixes that take the plane around a storm. The planner 
can also view the detailed spreadsheet and look at fuel consumption, winds and turbulence for the 
next flight segment in order to decide whether to change altitude. It would also be possible to 
support the case-based reasoning solution by providing the planner with access to already tried 
solutions that have been successful or by constraining the computer to consider only ATC 
preferred routes in generating suggestions. The planner could then make minor modifications to 
these successful plans. 

Planning Model 2. Under Planning Model 1, the planner doesn’t worry too much about a 
complete path to his/her destination. He/she simply finds an amendment that solves the immediate 
problem and assumes that the remainder of the solution can be worked out when the time comes. 

We also saw cases where the pilots in our simulator study worked out the entire flight plan after 
proposing an amendment In such cases, planning was again very decentralized. No one asked: 
What’s best for the whole system? ATC did, to some extent, look at the interactions among planes 
and put constraints on the solutions. The flight crew simply searched for a solution for their own 
plane that met these constraints. 

There are seveipl ways in which a flight planning aid could support such planning. The first 
would be to provide the raw data and calculations (winds, turbulence, fuel consumption, etc.) 
necessary for the human planner to work out a complete solution using forward search methods. 
The second would again mimic case-based reasoning approaches, borrowing from already 
generated solutions used by other planes or generated by ATC. 

The third approach mimics current human-to-human interactions. In our simulation studies, we 
sometimes saw pilots use heuristics like “try to fly upwind of the storm” to develop fairly abstract 
plans and then let ATC or Dispatch work out the details. They would say things like: 



“Can you find us a route north of this storm?” 

“We need a new destination airport.” 

By supporting planning at different levels of abstraction, out testbed mimics some aspects of this 
human-to-human interaction. Additional features worth considering based on this model, 
however, include allowing the human planner to specify a goal or constraint (such as “find a route 
that gets me to my destination within 10 minutes of my scheduled arrival time” or “find me an 
alternate destination” or “find a good airport that I can reach within 30 minutes” or “find an airport 
that I can reach and still have adequate holding fuel.”) 

Planning Model 2 - Discussion. Planning Model 2 has two important characteristics. First, like 
Planning Model 1, the planner doesn’t worry (too much) about finding global solutions that lead to 
good overall solutions for all of the air traffic. Second, unlike Planning Model 1, the planner 
works out the entire remainder of his own flight. He/she uses a much longer planning horizon. 

Finally, as discussed above, our simulation data suggests that pilots use a variety of solutions to 
generate such plans. They use forward search methods; they use case-based reasoning; they plan 
at higher levels of abstraction and then offload planning to another agent (Dispatcher or ATC) by 
merely specifying a goal or constraint All of these methods have potentially important 
implications for building computer aids. 

Planning Model 3. Planning Models 1 and 2 involved looking for solutions from a 
decentralized perspective. The planner (the flight crew in this case) looked for a plan that was 
good for him/her without directly considering whether that plan was good from a global 
perspective. (The global perspective was still partially considered by ATC when deciding whether 
to approve a requested change in altitude, etc.) 

A third planning model that we have seen in use involves explicitly considering the bigger picture. 
Such planning is currently done by ATC and Dispatch. This model is typically invoked when there 
is some large, systemic disturbance (a line of thunderstorms, airport closings, etc.). In such a 
case, ATC and Dispatch look for broader solutions that consider the overall implications for all of 
the air traffic (or, in the case of Dispatcher, at least that airline’s air traffic). At present, this global 
planning involves both elements of cooperation and competition. Dispatch would like to find good 
solutions for his/her airline. ATC would like to find good overall solutions. Furthermore, each 
group has access to different information. 



From the flight crew’s perspective, such planning often takes the form of case-based reasoning. 
The crew is informed that ATC has developed a preferred alternate plan for planes along that path, 
or that Dispatch has a recommendation. The crew then evaluates this plan to ensure that it is 
acceptable to them. 

Concept 7 - Discussion. Above, we describe a variety of planning “models” and methods 
that we have observed in use under current circumstances. These observations are of considerable 
importance, as it is likely that an effective cooperative system should support such alternative 
“models” and planning methods. 

Concept 8, Use graphics to enhance perceptual processes, helping the planner to 
“see” the important patterns instead of making him/her laboriously 
“reason” about the data in order to infer their presence. 

The attention literature makes a distinction between automatic recognition processes and controlled 
processes. Larkin and Simon (1987) suggest this concept can be fruitfully applied to designing 
aids for problem-solving. 

The most interesting application of this concept to flight planning is with the map display. By 
allowing the planner to observe the plane moving along its route, viewing concomitant changes in 
the weather predictions, the planner may find it much easier to judge trends and note important 
patterns. 

The detailed spreadsheet illustrates another simple application of this concept By embedding into 
the spreadsheet graphics identifying the current flight plan, a fuel efficient plan and maximum 
altitudes, it should be much easier for the planner to identify pertinent data and make comparisons 
at different altitudes. As another example, cloud TOPS could be embedded graphically in the 
spreadsheet. 

Concept 9. When using graphics, provide a “natural” mapping between the 

features of the display and the corresponding concepts or real-world 
objects. 

The map display is an obvious application of this concept The detailed spreadsheet is also 
consistent with it, however. The spreadsheet depicts the horizontal movement along jetways as a 
horizontal sequence of cells on the spreadsheet. Each successive column represents the next 
navigational fix or jet route in sequence. (An interesting conflict arises, though, when the plane is 
flying east to west. Should the sequence on the spreadsheet now go from right to left to be 



consistent with the orientation of the map display? Probably not) 


The altitude information at the bottom of the spreadsheet is also consistent with this principle. 
Higher altitudes for a flight segment are represented as higher cells in the spreadsheet 

There is also another inconsistency with this principle. The length of flight segments is not 
reflected at all in the graphics on the detailed spreadsheet. All spreadsheet columns are equally 
wide, even though the flight segments they represent differ in length. We have experimented with 
displays where segments lengths were drawn to scale. Segment lengths differ greatly, however, 
and our judgement was that it would be better to tradeoff in favor of compactness of the display 
(allowing the planner to see more flight segments at a time) rather than having pictorial realism. 

Concept 10. Consider distributing the problem-solving to simplify the tasks for 
individual participants. 

At present, there are several parties involved in flight planning. All three parties (the flight crew, 
ATC and Dispatch) play a major role in detecting problems that require replanning. The flight crew 
and ATC typically do the replanning for minor changes. Dispatch is more likely to play a role in 
dealing with major deviations from the original flight plan, in cooperation with both the flight crew 
and ATC. 

Such a distribution of flight planning activities allows the different parties to deal with different 
aspects of the flight planning problem. Such task decompositions need to be considered when 
deciding who should have access to what information and computer aids. 

Concept 11. Consider including redundancy in a distributed problem-solving 

environment to increase the likelihood that good solutions will not be 
overlooked and that bad solutions will not be accepted. 

In addition to reducing the cognitive load by distributing tasks among different parties, such shared 
problem-solving may benefit from intentional or chance occurrences of redundancy. Dispatch, for 
example, may notice that a flight amendment proposed by the flight crew leaves very little holding 
fuel and recommend finding an alternative plan. 

In designing the planning environment, we may want to use computers and advanced 
communication capabilities to enhance such intended and incidental redundancy. There may be 
data and information that we want to deliberately present to multiple parties. This may include 
presenting the computer’s conclusions, explorations and warnings to both the flight crew and 



Dispatch (and in some cases, to ATC as well). 


The literature on human error discusses such things as the generation of false assumptions (Fraser, 
Smith and Smith, 1992; Smith, Giffin, Rockwell and Thomas, 1986), and fixations on incorrect 
hypotheses or unwise solutions. In our simulation study we saw one example of such behavior. 
One crew appeared to fixate on Toledo as an alternate destination after Detroit was closed. 

Initially, it appeared to be a reasonable alternative, but given the questionable weather in the area 
and the progressively lower fuel levels, it was a very dubious choice to commit to while over 
Gopher. The crew never asked: Do we have enough fuel to go elsewhere if the weather at Toledo 
turns bad (or if air traffic congestion develops)? Appropriate aids to enhance distributed problem- 
solving might help reduce such errors. 

Concept 12. Design assuming that novel situations will arise that will make 
invalid certain inferences and conclusions made by the computer 
system. 

It is clear that knowledge-based systems and optimization programs have limited scope. It is quite 
probable that situations will arise that were not anticipated by the system designers. 

One solution is to provide the computer system with explicit error detectors (Smith, Fraser, et al., 
1991) and with metaknowledge. To the extent that the computer knows what it does and does not 
know, it will be better able to detect situations where it is “over its head.” This solution simply 
reduces the likelihood that the computer will unknowingly generate a questionable plan. There is 
still the likelihood that the system designers will leave out important metaknowledge to detect some 
novel situations. 


A second solution, therefore, is to keep people actively engaged in the planning activities, and to 
attempt to ensuje that they consider important data as well as recommendations made by the 
computer (or another person). This requires careful consideration of the roles of various agents 
(human and computer) as well the design and distribution of data displays. It is not enough to 
keep people “in the loop.” The system design must help to induce them to ask the right questions 
and view the right information at the right time. 


Concept 13. Try to predict the errors that components of the system, individually 
or jointly, could make. Try to design the overall system to prevent 
errors. Equally important, try to design the system so that errors 
(including those that haven’t been predicted) are likely to be caught 
or, failing that, so that their impacts are not serious. 



In our interviews and in our simulator studies, the most serious problems in planning seem to 
result from a combination of five factors: 


1 . Using a short planning-horizon to solve some immediate problem (thus failing to 
consider long-run implications); 

2 . Failing to detect a problem and/or discard the current plan early enough, while there are 
still many alternative options available; 

3 . Experiencing the occurrence of a series of events that, taken together, seriously threaten 
the plane’s safety, even though each one alone would normally be a minor problem; 

4 . Failure to look at the right data or to adequately consider its implications (such as the 
uncertainty associated with a weather forecast) when evaluating a proposed plan; 

5. Failure to even consider a potentially superior plan because of the fallibility of the 
heuristics used in generating a plan. 


In most cases, these problems lead to unnecessary costs, delays or discomfort. Once in a while 
they can result in serious hazards, however. 


Under Concept 7, we described a model of planning in which planning is very localized, in which 
the planner finds solutions to the immediate problem without considering in detail the implications 
for later in the flight This form of planning assumes a “friendly” world, where there are 
numerous alternatives to select from to solve the next step in developing the plan. Under such an 
assumption, there is no great need to look beyond solving the immediate problem. 

In flight planning, the assumption of a “friendly” world is normally quite viable. The plane has 
reserve fuel, keeping many options open. The plane can land somewhere else if fuel, weather, etc. 
make this necessary. Finally, the pilot can request priority clearances if the situation is becoming 
sufficiently difficult, thus gaining additional options. 


Occasionally, however, the flight crew finds itself in a less “friendly” world. Based on our 
interviews, this* seems to arise for one of two reasons: 

1 , The plane encounters a series of problems that each requires flight amendments and 
uses up extra fuel. The solution to each problem taken alone is quite reasonable, but, 
taken together, fuel levels get unacceptably low. Thus, by failing to consider a longer 
planning horizon, and by failing to anticipate potential “worst case” possibilities, the 
planner ends up in a situation where he/she has few good options left; 

2. The planner “fixates” on his/her current plan too long, failing to notice that the other 
options are disappearing (due to low fuel). If the “worst case” arises and they can’t 
complete their current plan, they are in a difficult situation. 



Solutions. One solutions would be to make the world “friendlier.” The obvious (but 
expensive) way to accomplish this would be to require greater fuel reserves or reduce air traffic 
levels. A second would be to develop computer aids that help the planner to detect problems 
earlier, to view and appropriately interpret available information, to use a longer planning horizon 
and to anticipate possible “worst case” situations. A third would be to develop aids that monitor 
the situation and warn the planner when the number of options is becoming dangerously low. A 
fourth would be to facilitate distributed planning on the assumption that Dispatch, for example, 
might be less likely to share a fixation that a flight crew has developed (or vice versa). 

Conclusion 

Technological and conceptual advances in the design of knowledge-based systems, in optimization 
methods and in telecommunications offer powerful tools for improving performance in complex 
systems. In applying such technologies, however, we must identify the true problems and needs 
of the application area, and understand the limitations of the available technologies. 

An important conceptual approach for the development of computer-based cognitive tools or aids is 
to explicitly design systems to enhance cooperative problem-solving. This approach starts with the 
assumption that, for both economic and technological reasons, there are many areas where 
complete automation is unlikely to provide an acceptable solution. Consequently, if we are to 
make effective use of current computer capabilities, we need to understand how to design cognitive 
tools that people can work with effectively. 

Above, we describe an effort to explore this conceptual approach using our Flight Planning 
Testbed to design aids for enroute flight planning. As part of the process of building this artifact, 
we have identified a number of general design concepts that proved useful in guiding design 
decisions. These design concepts, discussed and illustrated above, serve to point out possible 
ways to improve overall system performance by facilitating shared problem-solving. Further 
research is needed to evaluate such concepts empirically in order to assess their value. 
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Figure 1. Map display and associated controls for selecting information to display. 

(The actual display is color-coded.) 

Figure 2. Map display showing controls for asking the computer to find a new route subject 
to certain constraints. (The actual display is color-coded.) 

Figure 3. Spreadsheet display showing information associated with particular flight 
segments. (The actual display is color-coded.) 
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